Athentication of device users by gaze

ABSTRACT

A method includes obtaining a gaze feature of a user of a device, wherein the device has already been unlocked using a second feature, the gaze feature being based on images of a pupil relative to a display screen of the device, comparing the obtained gaze feature to known gaze features of an authorized user of the device, and determining whether or not the user is authorized to use the device based on the comparison.

RELATED APPLICATION

This application claims priority to U.S. Provisional Application Ser.No. 61/826,721 (entitled AUTHENTICATION OF MOBILE PHONE USERS BY GAZE,filed May 17, 2013) which is incorporated herein by reference.

BACKGROUND

With the increasing use of smartphones and tablets by large corporationsand defense agencies, there is increasing concern about the security ofsuch devices. The US Department of Defense, in particular, is interestedin actively authenticating mobile device users. This desire arises inpart because of the ineffectiveness of passwords and in part because thedevice may change hands after the rightful owner has unlocked it. Thereis a need, then, to authenticate the user of a mobile device as it isbeing used for its intended purpose.

In one prior device, a user may unlock a phone by gazing at images in aspecified sequence. The device is unlocked when eye tracking datamatches expected eye tracking data. One prior device purports to track auser's eyes to determine where to scroll. When a user reads text, andeye tracking indicates the last text on the page has been read, the nextpage of text may be automatically displayed.

SUMMARY

A method includes obtaining a gaze feature of a user of a device,wherein the device has already been unlocked using a second feature, thegaze feature being based on images of a pupil relative to a displayscreen of the device, comparing the obtained gaze feature to known gazefeatures of an authorized user of the device, and determining whether ornot the user is authorized to use the device based on the comparison.

The method may also be implemented via code stored on a computerreadable storage device.

In a further embodiment, a system includes a wireless communicationscapable device having a processor, a memory, and a display screen, acamera supported by the device and facing toward a user of the device toprovide video of a user's pupil to the processor, programming stored onthe memory, the programming adapted to cause the processor to identify agaze feature of the user, compare the identified gaze feature of theuser to known gaze features of the user, and confirming whether or notthe user is the user that is authorized and has passed a separatesecurity check to use the device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system using gaze information toauthenticate a user according to an example embodiment.

FIG. 2 is a flowchart illustrating a method 200 of authenticating a useras a function gaze information according to an example embodiment.

FIG. 3 is a block diagram of a computer system for implementing one ormore methods according to example embodiments.

DETAILED DESCRIPTION

In the following description, reference is made to the accompanyingdrawings that form a part hereof, and in which is shown by way ofillustration specific embodiments which may be practiced. Theseembodiments are described in sufficient detail to enable those skilledin the art to practice the invention, and it is to be understood thatother embodiments may be utilized and that structural, logical andelectrical changes may be made without departing from the scope of thepresent invention. The following description of example embodiments is,therefore, not to be taken in a limited sense, and the scope of thepresent invention is defined by the appended claims.

The functions or algorithms described herein may be implemented insoftware or a combination of software and human implemented proceduresin one embodiment. The software may consist of computer executableinstructions stored on computer readable media such as memory or othertype of storage devices. Further, such functions correspond to modules,which are software, hardware, firmware or any combination thereof.Multiple functions may be performed in one or more modules as desired,and the embodiments described are merely examples. The software may beexecuted on a digital signal processor, ASIC, microprocessor, or othertype of processor operating on a computer system, such as a personalcomputer, server or other computer system.

A front facing camera is among the sensors available on typical handheld devices, such as smartphones and tablets. While the front facingcamera is used today for authentication, such as a “face unlock”feature, such an unlock feature is not applied in an active way while adevice is already unlocked and in active use.

Once the phone is unlocked, there is no ongoing authentication.Moreover, face recognition has issues dealing with non-uniformillumination and may not handle appearance variations of the same person(say, a Halloween costume).

In order to address these issues, various embodiments activelyauthenticate a mobile device user based on their gaze patterns, asobserved by the front-facing camera, proximity sensor or wave sensor.Other sensors may also be used in further embodiments, and may utilizelower power than a camera. Visual attention differs as a function of theviewer's expertise in understanding what they are looking at. Given thatmobile devices can be customized by the owner, other users can bethought of as lacking expertise in understanding the device. The otheruser may not know where desired icons are located, or whether neededapps are installed on the device. As such, the rightful owner of amobile device should be more efficient than a non-owner, who may spendquite a bit more time scanning the display in search of a desired icon.

One embodiment includes software running on the mobile device. Imagesfrom the front-facing camera or other sensor are recorded, eye trackingis applied, and the recorded history of eye position as a function oftime is used as input to a verification module which returns a positiveresult when the eye motions are consistent with the owner's behavior,and a negative result when eye motions are inconsistent. In otherembodiments, eye position data may be recorded from ancillary sensorswhich are designed to capture gaze information with lower powerconsumption.

In one embodiment, adequate performance may be achieved by directclassification on the motion pattern. Further embodiments may utilizeextraction of features (locations of inflection points, etc.) may be amore successful method. In addition, temporal segmentation may beapplied to the eye motion history. By first de-composing the motionsequence into periods of known behavior (fixating, scanning, etc.),histogram-type features may be generated against which a machinelearning model (SVM, etc.) may be applied.

By tracking eye motions over time and correlating the motions to thecontext of user interaction with a display, including selection offeatures, multiple features may be extracted. Example features includelength of time looking at a point on the display screen, scanning speedand duration while reading text or identifying the location ofappearance of a feature selected by the user, anticipation of where afeature will occur—referred to as anticipatory gaze, and a multitude ofother features.

Anticipatory gaze for example may include the time from a click orselection of a feature to the time the user focuses on a point where thefeature will materialize on the display screen. The time may be negativeif the user looks prior to the appearance of the feature, or may bepositive if the user appears to spend time looking for the feature afterit appears. Still further features include a trajectory of the gaze, andvarious statistics regarding gaze, such as standard deviation of ahorizontal gaze, reading speeds, and various other statistics on whenusers look at different portions of the display screen.

The gaze pattern comparisons may be utilized in conjunction with othersecurity features, such as speaker recognition, how a phone is beingheld, passwords, and other biometric security features may be used. Thegaze pattern comparisons may be used after the user has already gainedaccess to the phone or other device using the other one or more securityfeatures.

FIG. 1 is a block diagram of adevice 100 having a processor 110, amemory 115, and a display screen 120 shown in cut away. The device maybe a mobile device such as a smart phone, a laptop computer, a desktopcomputer or other computing device. A gaze sensor 125 is supported bythe device 100 and facing toward a user of the device 100 to providegaze tracking information to the processor 110. The gaze sensor 125 maybe a camera in some embodiments. Programming 130 is stored on the memory115. The programming 130 is adapted to cause the processor 110 toidentify a gaze feature 135 of the user, compare the identified gazefeature of the user to known gaze features 140 of the user, and confirmwhether or not the user is the user that is authorized and has passed aseparate security check 145 to use the device 100.

In one embodiment, the gaze feature comprises a length of time lookingat a portion of the display screen, or alternatively a speed andduration of scanning. In a further embodiment, the gaze feature comprisean anticipatory gaze of the user, which may include a length of timefrom a user selecting a function to the time a user gazes in a properlocation where a feature will appear. The length of time is negative ifthe user gazes in the proper location prior to the feature appearing andpositive if the user gazes in the proper location after the featureappears. The gaze feature may also include a standard deviation of ahorizontal gaze. In still further embodiments the gaze featurescomprises reading speed.

FIG. 2 illustrates a method 200 of authenticating a user using gazetracking. The method 200 starts by obtaining, at 210, a gaze feature ofa user of a device after the device has already been unlocked using asecond feature. The gaze feature may be based on gaze trackinginformation relative to a display screen of the device. At 220, theobtained gaze feature is compared to known gaze features of anauthorized user of the device. At 230, method 200 determines whether ornot the user is authorized to use the device based on the comparison.

In one embodiment, multiple gaze features are obtained and compared onwhich to base the determination 230. The gaze feature may include one ormore of a length of time looking at a portion of the display screen, aspeed and duration of scanning, and an anticipatory gaze of the user.The anticipatory gaze feature may include a length of time from a userselecting a function to the time a user gazes in a proper location wherea feature will appear. The length of time is negative if the user gazesin the proper location prior to the feature appearing and positive ifthe user gazes in the proper location after the feature appears.

In a further embodiment, the gaze feature comprises a standard deviationof a horizontal gaze. The gaze feature may also include a reading speed.The reading speed may be derived from analyzing the gaze with respect toa number of words read during a known time period.

In yet further embodiments, the method 200 may be implemented inprogramming on a computer readable storage device when read and executedby processing circuitry.

FIG. 3 is a block schematic diagram of a computer system 300 toimplement a controller according to an example embodiment. The computersystem 300 may be a smart phone, tablet, personal computer, wearablesmart device, desktop computer or other computing device in variousembodiments. While many components are shown, different computingdevices may include only a subset of the components in some embodiments.

One example computing device in the form of a computer 300, may includea processing unit 302, memory 303, removable storage 310, andnon-removable storage 312. Memory 303 may include volatile memory 314and non-volatile memory 308. Computer 300 may include—or have access toa computing environment that includes—a variety of computer-readablemedia, such as volatile memory 314 and non-volatile memory 308,removable storage 310 and non-removable storage 312. Computer storageincludes random access memory (RAM), read only memory (ROM), erasableprogrammable read-only memory (EPROM) & electrically erasableprogrammable read-only memory (EEPROM), flash memory or other memorytechnologies, compact disc read-only memory (CD ROM), Digital VersatileDisks (DVD) or other optical disk storage, magnetic cassettes, magnetictape, magnetic disk storage or other magnetic storage devices, or anyother medium capable of storing computer-readable instructions. Computer300 may include or have access to a computing environment that includesinput 306, output 304, and a communication connection 316. In the caseof computer 300 being a wireless mobile device, the output 304 mayinclude a touchscreen, which may also operate as an input device. Theinput 306 may also include a gaze tracking sensor, such as a camera.Communication connection 316 may include a transceiver suitable formobile communications, such as cellular networks, local area networksand wide area networks, as well as hardwired connections. The computermay operate in a networked environment using a communication connectionto connect to one or more remote computers, such as database servers.

Computer-readable instructions stored on a computer-readable medium areexecutable by the processing unit 302 of the computer 300. A hard drive,CD-ROM, and RAM are some examples of articles including a non-transitorycomputer-readable medium. For example, a computer program 318 capable ofproviding a generic technique to perform access control check for dataaccess and/or for doing an operation on one of the servers in acomponent object model (COM) based system may be included on a CD-ROMand loaded from the CD-ROM to a hard drive. The computer-readableinstructions allow computer 300 to provide generic access controls in aCOM based computer network system having multiple users and servers.

EXAMPLES

1. A system comprising:

-   -   a device having a processor, a memory, and a display screen;    -   a gaze sensor supported by the device and facing toward a user        of the device to provide gaze tracking information to the        processor;    -   programming stored on the memory, the programming adapted to        cause the processor to:        -   identify a gaze feature of the user; compare the identified            gaze feature of the user to known gaze features of the user;            and        -   confirming whether or not the user is the user that is            authorized and has passed a separate security check to use            the device.

2. The system of example 1 wherein the gaze feature comprises a lengthof time looking at a portion of the display screen.

3. The system of example 1 wherein the gaze feature comprises a speedand duration of scanning.

4. The system of example 1 wherein the gaze feature comprise ananticipatory gaze of the user.

5. The system of example 4 wherein the anticipatory gaze featureincludes a length of time from a user selecting a function to the time auser gazes in a proper location where a feature will appear.

6. The system of example 5 wherein the length of time is negative if theuser gazes in the proper location prior to the feature appearing andpositive if the user gazes in the proper location after the featureappears.

7. The system of example 1 wherein the gaze feature comprises a standarddeviation of a horizontal gaze.

8. The system of example 1 wherein the gaze features comprises readingspeed.

9. A method comprising:

-   -   obtaining a gaze feature of a user of a device, wherein the        device has already been unlocked using a second feature, the        gaze feature being based on gaze tracking information relative        to a display screen of the device;    -   comparing the obtained gaze feature to known gaze features of an        authorized user of the device; and    -   determining whether or not the user is authorized to use the        device based on the comparison.

10. The method of example 9 wherein multiple gaze features are obtainedand compared on which to base the determination.

11. The method of example 9 wherein the gaze feature comprises a lengthof time looking at a portion of the display screen.

12. The method of example 9 wherein the gaze feature comprises a speedand duration of scanning.

13. The method of example 9 wherein the gaze feature comprise ananticipatory gaze of the user.

14. The method of example 13 wherein the anticipatory gaze featureincludes a length of time from a user selecting a function to the time auser gazes in a proper location where a feature will appear.

15. The method of example 14 wherein the length of time is negative ifthe user gazes in the proper location prior to the feature appearing andpositive if the user gazes in the proper location after the featureappears.

16. The method of example 9 wherein the gaze feature comprises astandard deviation of a horizontal gaze.

17. The method of example 9 wherein the gaze features comprises readingspeed.

18. A computer readable storage device having instructions to cause aprocessor to perform a method, the method comprising:

-   -   obtaining a gaze feature of a user of a device, wherein the        device has already been unlocked using a second feature, the        gaze feature being based gaze tracking information relative to a        display screen of the device;    -   comparing the obtained gaze feature to known gaze features of an        authorized user of the device; and    -   determining whether or not the user is authorized to use the        device based on the comparison.

19. The computer readable storage device of example 18 wherein multiplegaze features are obtained and compared on which to base thedetermination.

20. The computer readable storage device of example 18 wherein the gazefeature comprises a length of time looking at a portion of the displayscreen.

21. The computer readable storage device of example 18 wherein the gazefeature comprises a speed and duration of scanning.

22. The computer readable storage device of example 18 wherein the gazefeature comprise an anticipatory gaze of the user.

23. The computer readable storage device of example 22 wherein theanticipatory gaze feature includes a length of time from a userselecting a function to the time a user gazes in a proper location wherea feature will appear.

24. The computer readable storage device of example 23 wherein thelength of time is negative if the user gazes in the proper locationprior to the feature appearing and positive if the user gazes in theproper location after the feature appears.

25. The computer readable storage device of example 18 wherein the gazefeature comprises a standard deviation of a horizontal gaze.

26. The computer readable storage device of example 18 wherein the gazefeatures comprises reading speed.

Although a few embodiments have been described in detail above, othermodifications are possible. For example, the logic flows depicted in thefigures do not require the particular order shown, or sequential order,to achieve desirable results. Other steps may be provided, or steps maybe eliminated, from the described flows, and other components may beadded to, or removed from, the described systems. Other embodiments maybe within the scope of the following claims.

1. A system comprising: a device having a processor, a memory, and adisplay screen; a gaze sensor supported by the device and facing towarda user of the device to provide gaze tracking information to theprocessor; programming stored on the memory, the programming adapted tocause the processor to: identify a gaze feature of the user; compare theidentified gaze feature of the user to known gaze features of the user;and confirm whether or not the user is the user that is authorized andhas passed a separate security check to use the device.
 2. The system ofclaim 1 wherein the gaze feature comprises a length of time looking at aportion of the display screen.
 3. The system of claim 1 wherein the gazefeature comprises a speed and duration of scanning.
 4. The system ofclaim 1 wherein the gaze feature comprise an anticipatory gaze of theuser.
 5. The system of claim 4 wherein the anticipatory gaze featureincludes a length of time from a user selecting a function to the time auser gazes in a proper location where a feature will appear.
 6. Thesystem of claim 1 wherein the gaze feature comprises a standarddeviation of a horizontal gaze.
 7. The system of claim 1 wherein thegaze features comprises reading speed.
 8. A method comprising: obtaininga gaze feature of a user of a device, wherein the device has alreadybeen unlocked using a second feature, the gaze feature being based ongaze tracking information relative to a display screen of the device;comparing the obtained gaze feature to known gaze features of anauthorized user of the device; and determining whether or not the useris authorized to use the device based on the comparison.
 9. The methodof claim 8 wherein multiple gaze features are obtained and compared onwhich to base the determination.
 10. The method of claim 8 wherein thegaze feature comprises a length of time looking at a portion of thedisplay screen.
 11. The method of claim 8 wherein the gaze featurecomprises a speed and duration of scanning.
 12. The method of claim 8wherein the gaze feature comprise an anticipatory gaze of the user. 13.The method of claim 12 wherein the anticipatory gaze feature includes alength of time from a user selecting a function to the time a user gazesin a proper location where a feature will appear.
 14. The method ofclaim 8 wherein the gaze feature comprises a standard deviation of ahorizontal gaze.
 15. The method of claim 8 wherein the gaze featurescomprises reading speed.
 16. A computer readable storage device havinginstructions to cause a processor to perform a method, the methodcomprising: obtaining a gaze feature of a user of a device, wherein thedevice has already been unlocked using a second feature, the gazefeature being based on gaze tracking information relative to a displayscreen of the device; comparing the obtained gaze feature to known gazefeatures of an authorized user of the device; and determining whether ornot the user is authorized to use the device based on the comparison.17. The computer readable storage device of claim 16 wherein multiplegaze features are obtained and compared on which to base thedetermination.
 18. The computer readable storage device of claim 16wherein the gaze feature comprises a length of time looking at a portionof the display screen.
 19. The computer readable storage device of claim16 wherein the gaze feature comprises a speed and duration of scanning.20. The computer readable storage device of claim 16 wherein the gazefeature comprise an anticipatory gaze of the user, wherein theanticipatory gaze feature includes a length of time from a userselecting a function to the time a user gazes in a proper location wherea feature will appear.